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Abstract. We study absolutely periodic points and trajectories of Hamiltonian 
systems. Our main result is a necessary and sufficient for a Hamiltonian system to 
have the following property: if there exists one absolutely periodic trajectory then 
all trajectories are periodic with the same period. 



Introduction 

Periodic and absolutely periodic points of Hamiltonian systems play an impor- 
tant role in the study of spectral and semi-classical asymptotics for partial differen- 
tial operators (see, for example, [DG, PP2, SV2]). For instance, if the set of periodic 
points of the geodesic flow on a compact Riemannian manifold is of measure zero 
then the classical Weyl two-term asymptotic formula holds, which means, roughly 
speaking, that the eigenvalues of (A being the Laplace-Beltrami operator) 

are distributed uniformly on the real line. If this set has a positive measure then 
the spectrum of may contain clusters — contracting groups of eigenvalues of 

unusually high total multiplicity [DG, SV2]. Similar results have been also obtained 
for 

(i) general self-adjoint elliptic partial differential operators on manifolds with- 
out boundary, where one has to consider the Hamiltonian flow generated by 
the principal symbol [DG, SV2]; 

(ii) boundary value problems, where instead of the Hamiltonian flows one has 
to deal with so-called Hamiltonian billiards [I, SV2] ; 

(iii) semi-classical problems, where the principal symbol is defined in a different 
way and is not necessarily a homogeneous function [PP2]. 

General two-terms asymptotic formulae for the counting function of a partial 
differential operator contain an oscillating term, given by an integral over the set 
of periodic points and involving the period function [PP2, SV2]. These formulae 
often imply that the spectrum does contain clusters whenever the period function 
is constant [SV2]. Therefore it is essential to know 

(i) whether the set of periodic points is of measure zero and, if not, 

(ii) whether the period function is constant. 



The first author would like to thank the Department of Mathematics of King's College, London, 
where this work was performed, for its hospitality. The first author was supported by the Royal 
Society and the second author by the British Engineering and Physical Sciences Research Council, 
Grant B/93/AF/1559. Both authors are very grateful to D. Vassiliev for valuable discussions. 



2 



M. NOVITSKII AND YU. SAFAROV 



Unfortunately, there are very few results on these problems. Even in the simplest 
case of the standard Euclidean billiard in a convex smooth domain it is still unknown 
whether the set of periodic points is of measure zero. 

The main aim of this paper is to show how one can obtain results on the above 
mentioned problems with the use of theory of quasianalytic functions. We shall 
take advantage of the notion of absolutely periodic point introduced in [DG]. An 
absolutely periodic point can be roughly described as a periodic point at which all 
the derivatives of the Hamiltonian flow coincide with that of the identical map (see 
Definition 4.1). 

The set of periodic points may have a very complicated structure. In the generic 
case, the set of absolutely periodic points is much poorer. However, it is known 
that these two sets are of the same measure (see Lemma 4.4). Therefore it is 
tempting to find conditions under which there are no absolutely periodic points 
and, consequently, the set of periodic points is of measure zero. 

Sometimes such conditions can be formulated in terms of smoothness. In [SV2] 
the authors considered, in particular, the Hamiltonian systems generated by an- 
alytic homogeneous Hamiltonians. It was proved that in the analytic case the 
existence of one absolutely periodic point implies that all sufficiently close points 
are absolutely periodic. This allows one to find sufficient conditions for the set of 
periodic points to be empty. 

In this paper we use the scale of Carleman spaces, which contains the class 
of analytic functions as a particular case. This scale can be divided into two 
parts: quasianalytic and nonquasianalytic classes. Our main observation is that 
the technique suggested in [SV2] is also applicable in the quasianalytic case. We 
extend results obtained in [SV2] to the quasianalytic Carleman classes and construct 
counterexamples which imply that these results do not hold for nonquasianalytic 
Hamiltonians. Thus, we obtain necessary and sufficient conditions in terms of the 
Carleman spaces. 

The paper is organized as follows. In Section 1 we define the Carleman classes 
and recall some results, including a criterion of quasianalyticity. Section 2 and 3 are 
devoted to definitions and basic results concerning Hamiltonian flows and billiards. 
In Section 4 we define periodic and absolutely periodic points and trajectories and 
formulate our main results (note that Lemma 4.2 is new even in the analytic case). 
These results are proved in Section 6. In Section 5 we give an explicit description 
of geodesies on a surface of revolution, which we use later in Section 5 in order to 
construct the counterexamples. 

As we have mentioned before, the results on periodic and absolutely points have 
applications in spectral theory. Without getting into details, we formulate a simple 
corollary of our Corollary 4.6 and [SV2, Theorem 1.6.1]. 

Corollary. Let M he a compact quasianalytic Riemannian manifold with strictly 
convex boundary and N{X) be the counting function of \/—A , where A is the 
Laplace- Beltrami operator subject to Dirichlet or Neumann boundary condition. 
Then 

iV(A) = coA'^ + ciA'^-^ + o(A'*-^), A^+oo, (0.1) 

where cq and ci are the classical Weyl coefficients. In particular, (0.1) is valid for 

ihp iiQiinl T.n/nlnrinn nn n rnnitpnT Anm nin iniih niinQinnnliii/ir sirrrilii mnnpnr hniinrlnrii 
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1. Carleman spaces Cirrin) 

Let {mn}^i be a nondecreasing sequence of positive numbers satisfying the 
following conditions: 

(1) rrin grow faster than any power of n; 

(2) mn+i/mn < nC^ for some positive constant C ; 

(3) the sequence {ln(m„/n!)}^]^ is convex with respect to n. 

Definition 1.1. Let D C be a bounded domain. The Carleman space C{mn, D) 
is defined as the set of all functions u e C°°{D) such that 

sup 

where d^u are the partial derivatives of u and C is a positive constant which may 
depend on u but is independent of a. We say that u e C{mn) if tt G C{mn, D) for 
any bounded open subset D of its domain of definition. 

Example 1.2. If m„ = n\ then C{mn, D) coincides with the class of real analytic 
functions. 

Example 1.3. If s > 1 then the sequence = n^'^ satisfy the conditions (l)-(3). 
The space C{mn) defined by such a sequence is called the Gevrey class. 

In a similar way one defines classes C(m^) of vector- valued and matrix- valued 
functions. We shall take advantage of the following results (see [D, Section 5]). 

Lemma 1.4. Composition of C{mn) -functions is a C{mn) -function. 

Lemma 1.5. Let F{t, z) he a real C{mn) -function of variables t E M , z E R-^. // 
dtF{to, zo) 7^ then the local t-solution t*{z) of the equation F{t, z) — 0, defined in 
a neighbourhood of zq, belongs to C{mn)- 

Lemma 1.6. // the Cauchy problem for the a system of ordinary differential equa- 
tions 

^^=fit,z{t)), z(0) = zoeR^, 

has a continuously differentiable solution z{t]ZQ) and f e C{mn) then z{t]ZQ) also 
belongs to C{mn) as a function of (t, 2:0). 

In view of Lemma 1.4 one can define C(m„) -manifolds and C(m„ )-functions on 
C(m^)-manifolds. The scale of spaces C(m^) on real analytic manifolds was used 
for the study of the wave front sets of solution of linear differential equations with 
C(m„) -coefficients [H, Chapter 8] and some problems of function theory [D]. 

Remark 1.7. One can define the classes C{mn) assuming only that nin grow faster 
than any power of n. The sequences {rUn} satisfying the conditions (2) and (3) 
are said to be regular. Lemmas 1.4-1.6 were proved in [D] only for the regular 
sequences. It is quite possible that these lemmas hold true under less restrictive 
conditions (for instance, one can try to apply the regularization theory [M]), and 
then all our further results remain valid as well. 

Definition 1.8. The class C(m^) is said to be quasianalytic if every function 
u e C{mn) which has an infinite order zero is identically equal to zero. 
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Example 1.9. The analytic functions are quasianalytic. 

The following theorem gives a criterion of quasianalyticity for the classes C(m^). 

Theorem 1.10 [M]. C(m^) is quasianalytic if and only if the series X^^i^^n^^" 
is divergent. 

2. Hamiltonian flows 

1. Hamiltonian flows in IR^ x W^. Let h{x,^) be a real infinitely differentiable 
function on x W'', where x = (xi, . . . , Xd) and C = (Ci) • • • ) Cd) ^re d-dimensional 
variables. The systems of ordinary differential equations 

x*it;y,r]) = h{x*it;y,v)i^*it;y,v))i x*(0;y,r]) = y, 

t{t]y,v) = -hx{x*{t;y,r]),C{t;y,r])), t{0;y,v) = V, 

is said to be a Hamiltonian system, and the function h is called the Hamiltonian. If 
the first and second derivatives of h are uniformly bounded then the Hamiltonian 
system has a unique global solution (x*,^*) for every set of initial data {y,r]) e 
X (see, for example, [Ha]). 
A solution 

{x*{t),C{t)) := {x*{t;y,ri),e{t;y,v)), teR, (2.1) 

of the Hamiltonian system is usually interpreted as a trajectory in the phase space 
M^d = iRc' X originating from the initial point {y,r)). The group of shifts along 
the Hamiltonian trajectories is said to be the Hamiltonian flow. It is easy to see 
that the Hamiltonian h is constant along the Hamiltonian trajectories. Moreover, 
the Hamiltonian fiow preserves the canonical 2- form da; A d^ and, consequently, the 
canonical measure dx d^ on R^'^ [A,CFS] . 

2. Hamiltonian flows on manifolds without boundary. If we identify the 
phase space with the cotangent bundle T*]R'^ (which simply means that the 
choice of coordinates x determines, in a standard way, the choice of coordinates 
^) then the solution ^*{t;y,r]) behaves under change of coordinates as a covector 
over the point C* {t'l V : v) ■ Therefore the above construction can be generalized to 
the case where the Hamiltonian is a function on the cotangent bundle T*M over a 
smooth (i-dimensional manifold M without boundary. 

If M is a manifold and /i is a smooth function on T*M then the Hamiltonian 
equations are understood in local coordinates, x = (xi, . . . , Xd) and y — {yi, . . . , yd) 
being local coordinates on (or points of) M , ^ — {^i, . . . , ^d) and rj — {rji, . . . , rjd) 
being the dual coordinates on (or points of) the fibres T*M and T*M respectively. 
In this case the solution (2.1) is a smooth trajectory in T*M. As before, the 
Hamiltonian flow preserves the Hamiltonian, the canonical 2-form and canonical 
measure on T*M (which are defined as dx Ad^ and dx d^ in every local coordinate 
system). 

3. Homogeneous Hamiltonian flows. Let h he a function on T*M satisfying 
the following conditions: 

(1) h{x, > whenever C 7^ 0; 

(2) h is infinitely smooth outside = 0}; 

(3) h is positively homogeneous of degree 1 in ^, that is, h{x,X^) = Xh{x,^) 
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Clearly, such a function is not smooth at {C = 0}. However, we can consider the 
Hamiltonian trajectories and Hamiltonian flow generated by h on the invariant sets 

T'M := {{x,OeT*M :h{x,Oy^0} = { {x, e T* M : ( 0} 

and 

S*M := {{x,O^T'M:h{x,0 = l}. 

If conditions (l)-(3) are fulfilled then the Hamiltonian flow has the following 
additional properties (see, for example, [SV2]): 

(i) the solutions x* and ^* of the Hamiltonian system are positively homoge- 
neous in rj of degrees and 1 respectively; 

(ii) X* does not vanish, that is, the trajectory x*{t) C M cannot stop; 

(iii) the Hamiltonian flow preserves the canonical 1-form ^ • dx. 

In the theory of elliptic (pseudo) differential operators one usually deals with the 
homogeneous Hamiltonian flow generated by the principal symbol to the power 
1/m , m being the order of the operator under consideration (see, for example [MS, 
SV2]). 

Example 2.1. Let M be a Riemannian manifold and h{x,^) — \$,\x , where is 
the length of the covector ^ eT*M . Then x*{t) are geodesies and 

d 

Ckit) = J29jk{x%t))x*{t), 

where {gjk} is the metric tensor and x* — {xl, . . . is the tangent vector at 

the point x* . The homogeneous Hamiltonian flow generated by this Hamiltonian 
is called a geodesic flow. Note that it is more common to define geodesies with the 
use of the Hamiltonian 

where {g^*'} := {gjk}~^- Since the Hamiltonian is constant along the trajectories, 
this simply means that the parameter t is chosen in a different way; if h{x,^) = \^\x 
then the geodesies are parameterized by their length. 

3. Homogeneous Hamiltonian billiards 

Throughout this section we assume that the Hamiltonian satisfies the conditions 
(l)-(3) of subsection 2.3. The definitions and results quoted below can be found in 
[SVl, SV2]. 

1. Billiard trajectories. Let M be a smooth d-dimensional manifold with bound- 
ary and /i be a homogeneous Hamiltonian on T*M. Near dM we shall use special 

coordinates x = {x', xa) such that x' G W^~'^ , dM = {xd = 0} and Xd > for 
points in the interior of M. Then ^ = i^',^d), where ^d is the so-called conormal 
component of ^. 

Let (y, rj) e T'{M \ dM), that is, y ^ dM . Assume that, at some time t = r, 
the Hamiltonian trajectory (2.1) originating from (27,77) hits the boundary at some 
point 

( ^* ( — ^^ ^^^ ^ Til i\/r 
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(in other words, x*{t - 0) G dM). Then {x*{t - 0), ^*(r - 0)) is said to be a point 
of incidence. At a point of incidence we always have 

hA^%T-o),c{T-o)) < 0. 

The incoming trajectory is said to be transversal if 

hA^*{T-0),C{r-0)) < 0. 

Definition 3.1. We say that {x* {t + 0) , ^* {t + 0)) e Tqj^M is a, point of reflection 
if 

(i) h{x*{r - 0), C{r - 0)) = hix*{r + 0),C{r + 0)) , 

(ii) x*{r-0) = x*{r + 0), 

(iii) the covectors ^*(r— 0) and ^*(t+0) differ only in their conormal component, 

(iv) /i^^(x*(r + 0),r(r + 0))>0. 

Since h^^{x*{T + 0),^*(r + 0)) > 0, the Hamiltonian trajectory originating from 
the point (x*(r + 0),^*(t + 0)) is well defined. It is called a reflected trajectory. 

Remark 3.2. Note that, generally speaking, there may be several points of reflection 
and reflected trajectories corresponding to one point of incidence. 

Definition 3.3. The trajectory obtained by consecutive transversal reflection is 

called a billiard trajectory. 

A billiard trajectory originating from an interior point (y, rj) may not be deflned 
for alH G M for one of the following reasons: 

(i) at some moment the incoming part of the trajectory is not transversal, 

(ii) the trajectory experiences an infinite number of refiections in a finite time. 

In the first case the trajectory is said to be grazing and in the second case it is 
called a dead-end. 

2. Billiard flows. If in our special coordinates 

(a;*(r-0),r(r-0)) = ((a;*)', 0, (D', , 
(.;*(r + 0),r(r + 0)) = ((x*)', 0, (H', Cj") 

then the reflection law can be written as 

M(^*)'.o,(r)',c) = M(a:*)',o,(r)',a), 
/i^,((x*/,o,(ry,a) > 0. 

Definition 3.4. We say that the simple reflection condition is fulfilled if, for every 
(a;',^') G T'dM , the function h{x' ,0, , ■) has the only local (and hence global) 
minimum . We say that the strong simple reflection condition is fulfilled if, in 
addition, h^^^A^',0,^',^f)^0. 

If the simple reflection condition is fulfilled then 

I ^1 f\ tl t \ It tSt\ \Jt ^ TED 
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Therefore for every transversal incoming trajectory there exists the only reflected 
trajectory and we have 

Under the simple reflection condition one can define the group of shifts along the 
billiard trajectories, which is called the billiard flow. The billiard flow is deflned 
for alH e R on a set of full measure [CFS] (that is, the set of starting points of 
grazing and dead-end trajectories is of measure zero) and has the same properties 
as the Hamiltonian flow (see subsections 2.2 and 2.3). 

Example 3.5. The Hamiltonian h{x,^) = on a Riemannian manifold with 
boundary generates so-called geodesic billiard. In this case x*{t) are geodesies (see 
Example 2.1) and the reflection law takes the usual form: the angle of incidence 
is equal to the angle of reflection. Clearly, the geodesic billiard satisfles the strong 
simple reflection condition. 

3. Hamiltonian billiards with nonnegative Hamiltonian curvature. If the 

simple reflection condition is fulflUed then the function 

is said to be the Hamiltonian curvature of dM. Here ix' ,^') e T'dM and {•, •} are 
the Poisson brackets. 

Definition 3.6. We say that the Hamiltonian billiard (or billiard flow) is convex 
if the strong simple reflection condition is fulflUed and 

k(a;',O>0, y{x',O^T'dM. (3.1) 

Example 3.7. If M is a domain in the Euclidean space and h{x,$,) = \^\ then 
(3.1) is equivalent to the usual definition of convexity. 

In Section 6 we shall use the following result. 

Lemma 3.8 [SV2, Lemma 1.3.17]. In a convex billiard there are no grazing and 
dead-end trajectories. 

4. Periodic and absolutely periodic trajectories 

1. Homogeneous Hamiltonian and billiard flows. Throughout this subsec- 
tion we assume that 

(i) the Hamiltonian is defined on T'M and satisfies the conditions (l)-(3) of 
subsection 2.3; 

(ii) either dM = or the corresponding homogeneous Hamiltonian billiard 
satisfies the simple reflection condition. 

For the sake of convenience we shall regard homogeneous Hamiltonian flows on 
manifolds without boundary as a particular case of homogeneous billiard flows and 
assume that, by deflnition, the Hamiltonian flows on manifolds without boundary 
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Definition 4.1. Let T>0. A trajectory i/o, ^yo), 2/0, %)) and its start- 

ing point {yo,r]o) e T'{M \ dM) are said to be 

(1) T-periodicii {x*{T;yo,r]o),C{T;yo,r]o)) = {yo^rjo) ; 

(2) absolutely T-periodic if the function 

\x*{T;y,r,)-yf + \C{T;y,ri) - (4-1) 

of the variables (y, rj) G T'{M \ dM) has an infinite order zero at {yo, rjo) ; 

(3) absolutely (T, /) -periodic if they are absolutely T-periodic and the trajectory 
hits the boundary I times as t E (0, T) ; 

(4) periodic if they are T-periodic for some T > ; 

(5) absolutely periodic if they are absolutely T-periodic for some T > 

We shall denote the sets of periodic, absolutely periodic, absolutely T-periodic 
points and absolutely (T, Z)-periodic points lying in S*{M\dM) by H, H'*, Hj, and 
Ilj^i respectively. 

The following two lemmas suggest that the points lying in a path-connected 
component of 11 have a common period. However, we do not know whether it is 
true in the general case. 

Lemma 4.2. Let the manifold M and Hamiltonian h belong to a quasianalytic class 
C{mn) , and let VL be a path- connected subset ofU."^ . Then either Q n Ilj. i = % or 

n c n« ^ . 

Lemma 4.3. If is a smooth path in IT and the period T{y,r]) is a continuous 
function of {y, ry) € 7 then T{y, rf) is constant on 7 . 

Note that in Lemma 4.2 the set may be disconnected even if M is connected. 

Indeed, if there exist grazing or dead-end trajectories then, generally speaking, the 
billiard flow is well defined for t e [0, T] only on a subset of T'M , which may well 
be disconnected. 

The following lemma implies, in particular, that the set H \ is of measure 
zero. 

Lemma 4.4 [SVl, SV2]. The set of points which are T-periodic but not absolutely 
T-periodic for some T > is of measure zero. 

Theorem 4.5. Let M be a compact connected manifold and let the billiard flow 
generated by a Hamiltonian h be convex. If M and h belong to a quasianalytic class 
C{mn) then the existence of one absolutely (T, l)-periodic billiard trajectory implies 
that all trajectories are {T., I) -periodic. 

Theorem 4.5 was proved in [SV2] in the analytic case. If dM — then it takes 
the following form. 

Theorem 4.5'. Let M he a compact connected manifold without boundary. If M 
and h belong to a quasianalytic class C{mn) then the existence of one absolutely 
T-periodic Hamiltonian trajectory implies that all trajectories are T-periodic. 
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Corollary 4.6. Let M be a compact connected manifold, dM ^ 0, and let the 
billiard flow generated by a Hamiltonian h be convex. If M and h belong to a 
quasianalytic class C{mn) and k ^ then the set of periodic points is of measure 
zero. 

The next two theorems show that the quasianalyticity condition in Theorems 
4.5 and 4.5' cannot be removed. 

Theorem 4.7. // the class C{mn) is not quasianalytic then there exists a Rie- 
m,annian C{mn) -manifold M without boundary such that the geodesic flow on M 
satisfies the following conditions: 

(1) for some T the set Ily has a positive measure, 

(2) but there are nonperiodic trajectories; moreover, S*M\n. is a set of positive 
measure. 

Theorem 4.8. // the class C{mn) is not quasianalytic then there exists a Rie- 
mannian C{mn) -manifold M with boundary such that 

(1) the geodesic billiard on M is convex with k = 0, 

(2) for some T and I the set 11^ ^ has a positive measure, 

(3) but there are nonperiodic billiard trajectories; moreover, S*M \ Yl is a set 
of positive measure. 

One can generahze the definition of absolutely periodic points in the following 
way. 

Definition 4.9. Let T*{y, rf) be a smooth function defined in a neighbourhood of 
(yojVo)- We say that the point {yo,Vo) is absolutely T*-periodic if the function 

|a:*(r*(y,r7);|/,?7)-y|2 + |r(r*(^/,77);2/,?7)-?7|2 (4.2) 

of variables {y,r]) has an infinite order zero at (yo, r/o)- 

However, in the quasianalytic case this is equivalent to Definition 4.1. Indeed, 
if the manifold and functions h and T* are quasianalytic then the function (4.2) 
is also quasianalytic. If it has an infinite order zero at {yo, rjo) then it is identically 
equal to zero in a neighbourhood of {yo, t]o), which means that all points {y, rf) of 
this neighbourhood are periodic with period T*(j/, 77). Now Lemma 4.3 implies that 
T* = T*{yo,r]o), and therefore the point (j/o^^o) is absolutely T*(j/o, '7o)-periodic. 

2. Branching Hamiltonian billiards. If the simple reflection condition is not 

fulfilled then the corresponding billiard is said to be branching. In this case there 
may exist infinitely many billiard trajectories originating from a fixed point {y,r]) G 
T'{M \ dM), moreover, the set of these trajectories may well be uncountable. 
Therefore Theorem 4.5, as it is stated above, is unlikely to be true even for the 
simplest branching billiards. 

For homogeneous branching billiards it is possible to prove that the set of starting 
points of grazing trajectories is of measure zero [SV2]. But, even in the analytic 
case, the measure of the set of starting points of dead-end trajectories may be 
positive [SVl]. 

One can classify trajectories by the type of their reflections, introduce the notion 
of periodic and absolutely periodic trajectories and prove statements similar to 
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we have effective sufficient conditions for the set of starting points of dead-end 
trajectories to be of measure zero. To the best of our knowledge, the only result in 
this direction (without requiring simple reflection) was obtained in [VI] . 

3. Nonhomogeneous Hamiltonians. From the geometric point of view, the 
nonhomogeneous Hamiltonian flows are more diflicult to study because they do not 
preserve the canonical one form. If the Hamiltonian h is not homogeneous then one 
has to consider the restriction of the Hamiltonian flow to 

Sa := {{x,OeT*M :h{x,0 = X} 

for each flxed A separately. If A is not a critical value of h then Ea is a smooth 
(2d — l)-dimensional submanifold, and one can deflne the sets H C Sa and H" C Ea 
in the same way as for homogeneous flows (with (4.1) being considered as a function 
on Ea). 

In the nonhomogeneous case the problems discussed in Introduction become 
much more difficult. Not only may the answer depend on A, but also the structure 
of the sets H and H" on a fixed energy surface E> may be more complicated. The 
following simple observation shows that Lemma 4.3 does not necessarily hold for 
the nonhomogeneous flows. 

Example 4.10. Assume that zero is not a critical value of the Hamiltonian h and 
let hg :— g h, where is a smooth strictly positive function. Then hg vanishes on 
Eo and the Hamiltonian trajectories (a;*,^*) of hg lying on Eq are deflned by the 
equations 

x*g{t] y, v) = 9{x*g, Cg) hi^l^ Cg), x*g{0; y,r]) = y, 

igit, y, V) = -9{Xg, Cg) hccixg, ^*), ^*(0; y,7]) = ri. 

This implies that x*g{t;y,r]) = x*{f{t;y,r]);y,r]) and C*{t;y,ri) = C{f{t',y,v);y,v), 
where ^ 

f{t;y,ri)= I 9{x*g{s;y,ri),Cg{s;y,ri)) ds . 
Jo 

Therefore every point {y, 77) e H C Eq is periodic with respect to the Hamiltonian 
flow generated by hg and its periods T and Tg are related as follows 

T{y,r]) = f{Tg{y,rj);y,rj). 

Clearly, the period Tg may vary from one point to another even if T is constant. 

In [PPl, PP2] the authors proved an analogue of Lemma 4.4 and, under certain 
additional restrictions, the analytic version of Theorem 4.5' for a class of nonhomo- 
geneous Hamiltonians, including Hamiltonians of the form h{x,^) = |CP + 
The latter result is likely to remain valid in the quasianalytic case. 

5. Geodesic flows on surfaces of revolution 

In this section we consider the geodesic flow on a 2-dimensional surface of rev- 
olution provided with the standard metric. In the flrst subsection we write down 
the differential equations for geodesies in an explicit form. We use the arguments 
suggested by D. Vassiliev in [V2], where 2-dimensional analytic manifolds whose 
geodesies are closed with the same length were described (see also [B, Chapter 4]). 

T„ . ] — u — „j-,,J,, j-1 J- „f „u — ;„J;„ ;„j-„ 
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1. Differential equations for geodesies. Let — f<^<§, < (p < 27r . 

Consider a surface of revolution M cM.^ defined by 

qi = cos6siia.(p, q2 = cos6 cos(p, qs = y [1 + f (ip)]'^ — siia^ ip dip , (5.1) 

Jo 

where qk are coordinates in and f{9) is a smooth function such that 

(1) /(o)=o, 

(2) 1 + fiij) > I sinV'l for all i/j G (-f , f ) , 

(3) / can be extended to a smooth function on M which is even with respect to 
the points — ^ and ^ in some neighbourhoods of these points, 

(4) the Taylor expansions of the function [1 + /(V')]^ — sin^ ip at the points — |- 
and ^ start with terms of degree 2q- and 2q+ respectively, where q- and 
q+ are arbitrary odd positive integers. 

In view of (2), the surface M is of the same smoothness as the fimction / outside 
the poles {9 — ±f } • The conditions (3) and (4) imply that the same is true in a 
neighbourhood of the poles. Indeed, one can easily prove that, under conditions (3) 
and (4), M can be defined in a neighbourhood of a pole by the equation z = F{r) , 
where r = ^/ x'^ + y'^ and F is an even function which belongs to the same class 
C(mn) as / . 

Example 5.1. If / G C^{—^, ^) and / satisfies the conditions (1) and (2) then 
/ also satisfies (3) and (4). 

The geodesic flow on a 2-dimensional surface in M.^ can be interpreted as the 
motion of a particle, with velocity and mass equal to one, in the field of inertial 
forces. It is described by the Euler-Lagrange equations 

^{dL/dxi) - dL/dxi = 0, i=l,2, 
where Xi = Xi{t) , Xi = Xi{t) is the tangent vector and 

3 / ^ \ 2 

L = L{xi,X2,xi,X2) := ^l-T:Qkixi,X2)] 

k=l ^ ^ 

is the Lagrangian, Xi being coordinates on the surface (see, for example [DFN]). 
The Euler-Lagrange equations imply that the function 

Ii{xi,X2,xi,X2) := xi{dL/dxi) + X2{dL/dx2) - L{xi,X2tXi,X2) 

is constant along the trajectories {x{t),x{t)). Note that this function turns into 
the corresponding Hamiltonian if we replace Xi with g^'^iki where {g^^} is the 
metric tensor. 

If M is defined as above and xi := 9 , X2 <p then 

L{9,9,ip) = Ii{9,9,<p) = cos'' 9 + 9^1 + f{9)]\ 

and, by the second Euler-Lagrange equation, the function 



T la _ 



„„„2 a 
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is constant along the trajectories. 

Having found two motion integrals /i and I2 for the Euler-Lagrange equations, 
we can write down equations for the geodesies in an explicit form. Assume that 

(i) the starting point of a geodesic (<^*(t), lies on the equator, that is, 
^*(0) = and so 

h = (p*it) cos'^e*{t) = cp*{0); (5.2) 

(ii) the geodesic (^(p* (t) , 0* (t)) is parametrized by its length, that is, 

h = {if*{t)fcos^e*{t) + {r{t)f[i + f{e*{t))f = i. (5.3) 

Clearly, <^*(0) = cos a, where a e [0, f ] is the angle between the geodesic and 
equator at the starting point. Therefore we can rewrite (5.2) as 

, COSCK , ^, 

^ W = (5.4) 

Now (5.3) and (5.4) imply 

\f)*(f\\ - A/cos2^*(t)-cos^a 

Note that \0*{t)\ < a and therefore cos'^e*{t) > cos^ a . Indeed, if 9*{t) = then, 
in view of (5.3) and (5.4), |6'*(t)| = a . In particular, if 9*{t) has a local maximum 
at t+ and a local minimum at t- then 

e*{t+) = -e*{t-) = a. (5.6) 

The last observation together with the fact that {6*{t))'^ = 1 — cos^ a whenever 
6* (t) = imply that the function 9* (t) is periodic. Moreover, if 

r (ti) = r{t2) = r{ts) = o , ti < t2 < , 

and 9*{t) 7^ for t e (^1,^2) U (^25*3) then its period coincides with — ti . In 
view of (5.5) and (5.6) we have 

J _q; v cos^ 9 — cos^ a J -a V cos^ 9 — cos^ a 



where 
Therefore 

for alH e R , where 



m := lim+fi-9)). 



9*{t + T,{a)) = 9*{t) (5.7) 



/a 



f{9) cos 9 



d9 
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is the period of 6*{t) . 

Similarly, in view of (5.4)-(5.6), 



/a 
-a 



1 + ^ 



COS 9 V cos^ 9 — cos^ a 



Using 

1 d / / cos a sin 

c 



arctan 



cos 9 V cos^ 9 — cos^ a d9 \ \ cos^ 9 — cos^ a , 

we obtain 

^*{U)-ip*{tx) = 27r + i?(a), (5.8) 

where 



a. 



R{a) := 2cosa / '^^^^ = d9 . 

J -a cos 9 V cos^ 9 — cos^ a 

Since the function 9*{t) is (Q;)-periodic, (5.4) and (5.8) imply that for alH G M 

ip*{t + T4a)) = (p*{t) + R{a) (mod 27r) . (5.9) 

2. Absolutely periodic points. In the previous subsection we have proved that 
any geodesic starting at the equator can be written as {(^* (t) , 9* (t)) , where the 
functions (f*{t) and 9*{t) satisfy the differential equations (5.4), (5.5). If we iden- 
tify vectors and covectors as in Example 2.1 then the corresponding Hamiltonian 
trajectory has the form {(f*{t),9*{t),(p*{t),9*{t)). In particular, the equator is a 
27r-periodic geodesic (with 'f*{0) = 1), and the corresponding Hamiltonian trajec- 
tory is r(t) — {t + (/p*(0),0, 1,0) , t e R. The following lemma gives a necessary 
and sufficient condition for this trajectory to be absolutely 27r-periodic. 

Lemma 5.2. The Hamiltonian trajectory r{t) is absolutely 2n-periodic if and only 
if all even derivatives of the function f{9) vanish at 9 = , that is, 

/(2'=)(0) = 0, /c = 0,1,2,... (5.10) 



Proof. Let (^9*{t;9,(f,9,(fi),(fi*{t;9,(f,9,(fi)j be a geodesic starting at the point 

{cp, 9, (p, 9) and Q!*(^, (p) be the angle of intersection of this geodesic with equator 
(here we consider (f,9,(p,9 as independent variables). Assume, for the sake of 
definiteness, that (p = . We have to prove that the functions 

^%27v;^, 9,^,9)-^, 9*i27v;^,9,pJ)-9, ^* {27v; 9, ^Jy^ , 9* {2n- ^,9,^,9)^9 

have infinite order zeros at the point {0, 0, 1, 0} if and only if (5.10) holds true. 
According to (5.4) 

cos a* — ^p cos^ 9 (5-11) 

and, by (5.7), 

9 = 9*{T,{a*y,^,9,>p,9). (5.12) 
If the function 9*{2t{; 9, (p,9) ^ 9 has an infinite order zero at {0, 0, 1, 0} then. 
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{9 = 0,(p = 1} . This fact and (5.11) imply that T*(q;) — 27r has an infinite order 
zero at a = , which is equivalent to (5.10). 

Assume now that all even derivatives of / vanish at ^ = . Then the functions 
R{a) and T^{a*) — 2n have infinite order zeros at {9 = 0, (p = 1} , so (5.12) 
and (5.9) respectively imply that 6'*(27r; </?, 9, (p,9) — 9 and (/?*(27r; ip, 9, ip,9) — (p 
have infinite order zeros at {0, 0, 1, 0} . By (5.4) and (5.5) the same is true for the 
functions <^(27r; (p, 9, (fi,9) — (p and 9{27r; (p,9,(fi,9) — 9 . □ 

Lemma 5.3. If f is odd on an interval [— Q;o,ao]; < ao < f then all the 
geodesies intersecting the equator at an angle a < ao are 27r-periodic. In particular, 
if f is odd on [— 7r/2,7r/2] then all geodesies are 27r-periodic. 

Proof. Under conditions of the lemma 

T^{a) = 27r , R{a) = , Va e [— ckq, ckq] . 
Therefore the required result immediately follows from (5.7) and (5.9). □ 

6. Proofs 

1. The period function. Let {yo,r]o) be a T-periodic point of the homogeneous 
Hamiltonian or billiard fiow. We are going to define in a neighbourhood of this 
point a positive function t*{y, rf) with the following properties: 

(1) t*(2/o, ^7o) = T and t* is positively homogeneous in r] of degree 0; 

(2) t* belongs to the same Carleman class C{mn) as the pair {h, M); 

(3) Vy,n't* {y, rj) = if and only if the point (y, rf) is periodic with period t* (y, rj) , 

(4) Vy^rji* has an infinite order zero at (y, rf) if and only if the point (y, rf) is 
absolutely t*(y, ?7)-periodic. 

Note that the function t* is not uniquely defined by (1)~(4). Indeed, outside the 
set n we only assume (1) and (2). Moreover, if {y,rf) e 11 or {y,rf) e 11'* but the 
corresponding period does not coincide with t*{y,rf) then we do not impose any 
restrictions on the derivatives of t* at (y, 77) . 

Lemma 6.1. Let the manifold M and Hamiltonian h belong to a class C{mn) and 
let {yo,r]o) be a T-periodic point. Then there exists a positive function t* satisfying 
the conditions (l)-(4) in a neighbourhood of {yo,r]o) . 

The proof is based on the following technical lemma (see [SV2, Lemma 2.3.2] or 
[T]). 

Lemma 6.2. Under conditions of Lemma 6.1 there exists a coordinate system in 
a neighbourhood of y^ such that det^riiT; yo,Vo) 7^ . 

Note that in [SV2, T] the authors only proved the existence of C°°-coordinates 
for which the matrix det^r]iT;yo,r]o) is nondegenerate. However, it is clear from 
the proof that in the case of C(mn )-manifold one can find C(mn )-coordinates with 
the same property. 

Proof of Lemma 6.1. Let us choose coordinates as in Lemma 6.2 and consider the 
function 
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defined in a neighbourhood of the point (T; yojilo) ■ Using the fact that the Hamil- 
tonian and biUiard flows preserve the Hamiltonian and canonical 1-form on T'M, 
one can prove that in a neighbourhood of (T; yojTjo) 

^ri{t;x,y,r]) = <^ X = x*{t;y,r]) , (6.1) 

'^At;x,y,v)U^.^t,y,r,) = e{{t;y,v), (6.2) 
Mt;x,y,v)\x^x*{t;y,v) = -Kv^v) (6.4) 

(see [SV2, Sections 2.3, 2.4] for details). 
Define 

^'(t;y,77) := <^{t;y,y,r]). 
In view of (6.4), in a neighbourhood of (y, ry) the equation 

*(t; 27,77) = 

has the only t-solution t*{y,r]). Since the point {yo,r]o) is T-periodic we have 
^{t;y,r]) = 0, and therefore t*{yQ,riQ) = T . Clearly, the function t* is positively 
homogeneous in r/ and, by Lemma 1.5, it belongs to the class C{mn) . 

From (6.1)-(6.3) it easily follows that Vy,^\E'(t; y,r]) — if and only if the point 
{y,r]) is t-periodic, and V y^ri'^it^y^rf) has an infinite order zero at {yo,r]o) (as a 
function of {y,r])) if and only if the point (j/o^^o) is absolutely t-periodic (see [SV2, 
Section 4.1] for details). Differentiating the identity 

*(r(y,r/);y,r/) = 
in y and rj and taking into account (6.4), we see that Vy^r]^{t; y, t7)|^^^*(-j^ = if 
and only if Vt*(y, -q) = , and dyd!^^{t; yo, ^o)|^_^,(y »7 ) ~ nonzero a, (3 

if and only if 'Vt*{y,r]) has an infinite order zero at {yo,r]o). This implies (3) and 
(4). □ 

2. Proof of Lemma 4.2. Let (j/0)^o) £ n^,^ . Since we consider only transversal 
reflections, the billiard trajectories (a;*(t; y, ry), ^*(t; ?])) arc well defined for all 
t G [0,T] and all {y,r]) lying in a neighbourhood of {yo,r]o)- Moreover, every 
trajectory starting in this neighborhood hits the boundary / times as t G [0, T] . 

Let t* be the period function defined on a smaller neighbourhood of (yo, Vo) ■ In 
view of (4) the gradient of this function has an infinite order zero at {yo,r]o) . Under 
conditions of the lemma t* is quasianalytic, and therefore it is identically equal to 
T in this neighbourhood. Thus, every point {y, rf) e ^ has a neighbourhood 

U{y,^) c , . 

Let 7 be a path in Vt and, for (y, -q) E , let T{y, rf) :— min{T : {y, rf) G H^} . If 
(y(n)^ ^(n)) _^ (^y^ ^) T(y("), r/(")) ^ T as n ^ oo then, obviously, (y, ry) G . 
This implies that the function T(j/, 77) takes its minimal value on 7 at some point 
(yo, ^0) ^ 7. Consider the set 7 fl 11^^^ . By the above, this set is open in 7 . On 
the other hand, since the restriction of the billiard flow 

(y,ry) ^ (x*(To; y, ry), r(To; y, ry)) 

to a neighbourhood of 7 is a smooth map, the set 7nn^^ is closed in 7 . Therefore 
7 C n^^^ . Since any two points of VL can be joined by a path, this implies that 

^ TT ; J — ] ^ TT -L (h 
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3. Proof of Lemma 4.3. Under conditions of the lemma, in a neighbourhood of 
every point (yo, r/o) the function T{y, rj) coincides with the restriction of the period 
function t* to 7 . Now (3) imphes that T{y, rj) is locally and hence globally constant 
on 7 . 

4. Proof of Lemma 4.4. The lemma immediately follows from the fact that the 
set of zeros of Vj,,^ is of the same measure as the set of infinite order zeros. 

5. Proof of Theorem 4.5. By Lemma 3.8, under conditions of the theorem the 

trajectories {x*{t;y,ri),^*{t-,y,r])) are well defined for all t G M and all (y, ry) G 
T'(M \ DM) . Let (yo, Vo) e ^t,i and 7 be a path in T'(M \ dM) from (yo, Vo) 
to another point {y^r]). In the same way as in the proof of Lemma 4.2 one can 
show that 7 C 11^ i . Since the set T'{M \ dM) is path-connected, this implies that 
n^.^^ =T'{M\ dM) . 

6. Proof of Corollary 4.6. In view of Theorem 4.5 it is sufficient to show 
that, given I and T we can find at least one trajectory which is not (T, /)-periodic. 
If k(x',^') 7^ then, by choosing the starting point close to (a;', 0, 0), we can 
construct a trajectory which experiences arbitrary many refiections in any given 
time (see proof of Lemma 1.3.34 in [SV2]). This proves the corollary. 

7. Proof of Theorems 4.7 and 4.8. Every nonquasianalytic class C(m„) con- 
tains nonnegative C^-functions [M]. Let / G C{mn) n C^(R) , / > and / = 
outside the interval (^, 1) • Consider the surface of revolution M defined by (5.1). 
Then, according to Lemma 5.3, all the geodesies intersecting the equator at an 
angle a <\ are 27r-periodic, which implies that the set 112^ is of positive measure. 

On the other hand, in view of (5.7) and (5.9), a geodesies intersecting the equator 
at an angle a is periodic only if 

kR{a) = 2/ccosa / '^^^^ = d^ 

J -a COS 9 V cos^ 9 — cos^ a 

= 2k cos a [ ■^^^^ =d9 = (mod 27r) (6.6) 

Jo cos 9 V cos^ 9 — cos^ a 

for some integer k . Since / has an infinite order zero at a = | and cos a is 
a decreasing function on (0, ^) , the function R{a) is strictly decreasing on an 
interval [^,0;'] , a' G (^,^] . Therefore, for each A;, (6.6) can only be true for a 
finite number of points a G ■ This implies that the set of periodic points 

corresponding to the trajectories with o: G [|, a'] is of measure zero. The measure 
of the set of starting points of all Hamiltonian trajectories with a G [|, a'] is not 
zero, so S*M \ 11 is a set of positive measure. 

In order to prove Theorem 4.8 it is sufficient to notice that our surface is sym- 
metric with respect to the plane {x = 0} . Therefore the billiard trajectories on 
the manifold with boundary M_|_ = {{x^y, z) E M : x > 0} behave in the same 
way as the geodesies on M ; namely, the reflected trajectories of the billiard flow 
are obtained by the reflection with respect to the plane {x = 0} of the parts of 
geodesies lying in the second half of M . This implies that, for the function / 
described above, 1127^ 2 and S*M \Il are sets of positive measure. 
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